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LisC  of  Tables 


Table  1.  Predictor  and  predlctand  categories  which  specify  the  dummy  variables 
used  in  GEM.  Shown  under  the  index  column  are  the  left-out  categories  not 
included  because  of  redundancy. 

Table  2.  Quantities  derived  for  the  designated  dummy  variables;  the  number  of 
times  each  category  occurred  in  the  sample  (IZ),  the  number  of  times  each 
predictor  occurred  when  it  was  followed  by  the  lowest  cloud  hit  ten  minutes 
later  (EYZ),  and  the  regression  coefficient  for  each  predictor  when  lowest 
cloud  hit  was  the  predictand  (Ajq). 

Table  3.  Brier  scores  for  a  10-minute  forecast  of  each  specified  predictand 
using  the  global  climatological  probability,  the  conditional  persistence 
probability,  and  GEM  based  on  the  developmental  sample  of  468408  cases. 

Dashes  denote  inapplicability. 

Table  4.  A  summarization  of  the  comparative  skill  between  GEM,  "G" ,  and 
persistence,  "P",  (ties  are  denoted  by  "T")  for  appropriate  scores  at 
projections  of  10,  20,  ...,  60  minutes  with  a  tally  for  each.  Percent 
correct  (PC),  Heidke  skill  score  (H),  and  threat  score  (T)  are  shown  for  the 
predictands  listed  in  section  2.2,  except  precipitation  occurrence. 


1 .  INTRODUCTION 


A  recent  analysis  of  rotorcraft  operations  (Taylor  and  Adams,  1985)  indicates 
that  the  "typical  flight  mission  consists  of  Point  A  to  Point  B  flights  that 
average  22  minutes  in  length,  incorporating  5  interim  stops  for  a  total  round- 
robin  flight  averaging  1  hour  48  minutes."  The  granularity  (geographic  and 
time)  of  weather  information  for  rotorcraft  operations  is  very  small  in  compari¬ 
son  with  what  is  required  for  fixed  wing  operations.  Thus,  the  rotorcraft  pilot 
has  little  interest  in  mid-range  (4-6  hours)  weather  forecasts.  What  is  needed 
is  short  range  (10-120  minutes)  forecasts  for  short  range  distances.  The  unique 
characteristics  of  rotorcraft  allow  them  to  land  at  places  where  airplanes  can 
not.  Many  of  these  landing  areas  have  low  density  traffic  which  does  not 
justify  a  weather  observer  much  less  a  forecaster.  The  rotorcraft  community  is 
extremely  Interested  in  an  automatic  weather  sensor  and  an  associated  system  for 
short  term  weather  forecasting.  It  is  for  this  reason  that  the  FAA  is  sponsor¬ 
ing  the  NWS  effort  described  in  this  report. 


A  statistical  technique  for  predicting  the  probability  distribution  of  all 
surface  weather  elements  inside  one  hour  is  called  GEM  for  a  Generalized 
Equivalent  Markov  model.  It  uses  only  the  current  local  automated  surface 
weather  conditions  as  predictors.  From  these  probability  distributions, 
categorical  predictions  are  made  for  each  automated  surface  weather  element. 

The  technique  is  a  Markov  procedure  which  is  briefly  described  in  the  following 
quotation  from  William  Feller  (1950): 


"In  stochastic  processes  the  future  is  never  uniquely  determined, 
but  we  have  at  least  probability  relations  enabling  us  to  make 
predictions  ....  The  term  "Markov  process"  is  applied  to  a  very 
large  and  important  class  of  stochastic  processes  ....  Concep¬ 
tually,  a  Markov  process  is  the  probabilistic  analogue  of  the 
processes  of  classical  mechanics,  where  the  future  development  is 
completely  determined  by  the  present  state  and  is  independent  of 
the  way  In  which  the  present  state  has  developed  ...  in  contrast 
to  processes  ...  where  the  whole  past  history  of  the  system 
influences  its  future." 


GEM  is  a  multivariate  linear  regression  system  in  which  all  variables,  both 
predictors  and  predictands,  are  zero-one.  It  is  Markovian  in  that  it  uses  only 
the  most  recent  observation  of  the  automated  surface  weather  elements  to  predict 
the  probability  distribution  of  those  same  automated  weather  elements.  In  the 
original  hourly  GEM  approach,  an  interative  scheme  was  employed.  For  such  short 
time  projections  as  one  minute,  the  iteration  scheme  has  been  replaced  by 
equations  which  predict  directly  to  the  desired  projection  time  of  interest. 


The  mathematical  model  is  given  in  Section  2.  Section  3  presents  results 
comparing  GEM  with  persistence.  Section  4  gives  a  summary  and  discusses  future 
plans. 


2. 


TECHNIQUE  DEVELOPMENT 


This  section  describes  the  procedure  from  the  mathematical  model,  through 
data  preparation  and  statistical  analyses,  to  a  discussion  of  a  nonlinear 
prediction  method.  The  reader  is  referred  to  a  NOAA  Technical  Report  for 
further  details  (Miller,  1981). 


2.1.  Mathematical  Model 


Assumed  given  are  measurements  on  a  set  of  Z  ,  Z  ,  ...,  Z  predictor 

12  p 

variables  and  a  set  of  Y,  ,  Y„  .,•••,  Y  .  predictand  variables  at  projection 

l,t  2,t  q , t 

time  t  for  a  group  of  N  observations.  The  problem  of  multivariate  regression 
is  to  construct  a  set  of  Q  linear  functions 
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which  have  the  property  that  the  sum  of  the  squares  of  the  errors 
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are  as  small  as  possible.  That  is,  the  problem  is  to  determine  values  of  the 

a  *s  (q  *  1,2,  ...,  Q:  p  *  1,2,  ...»  P)  which  minimize  the  quantities 
q.p.t 

2 

eq  ( q  =  1  ,  2,  ...,  Q ;  ta10,  20,  ...,  60) 


This  is  done  by  taking  the  partial  derivatives  of  Eq.  (2)  with  respect  to  the 
unknown  a’s,  setting  each  derivative  equal  to  zero,  and  then  solving  for  the 
a's.  The  process  yields  a  set  of  normal  equations  which  can  be  written  in 
matrix  notation  as  (underlining  signifies  a  matrix  or  vector): 

A  *  (Z’ZT^Z'Y  )  (t  -  10,  20,  ...,  60)  (3) 

— t - - t 

Expressed  statistically  this  is  the  multivariate  linear  regression  of  the  Y's 
on  the  Z’s  (Tatsuoka,  1971,  pp.  26-38).  In  GEM,  the  Y  values  are  advanced  by 
10,  20,  30,  40,  50,  and  60  minutes  from  the  corresponding  Z  values.  Thus 
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Once  the  A  's  have  been  determined,  they  can  then  be 
value  of  £  at  each  10-minute  time  step,  given  a  set  of 
step  (lower  case  values  denote  new  forecast  Y  based  on 


used  to  estimate  the 
z_  values  at  a  zero  time 
new  observation  Z): 


"  io  ^  Ct  =  10,  20,  ...  ,  60) 


(A) 


2.2.  Data  Preparation 


Data  collection  began  at  the  National  Weather  Service's  Techniques  Develop¬ 
ment  and  Test  Branch  location  at  Sterling,  Virginia,  in  April  1984.  The 
following  weather  elements  are  observed  once  a  minute  by  equipment  similar  to 
the  FAA's  Automated  Weather  Observing  System  (AWOS).  The  elements  are: 

o  Lowest  cloud  hit 
o  Second  cloud  hit 
o  Third  cloud  hit 
o  Fourth  cloud  hit 
o  Visibility 
o  Station  pressure 
o  Temperature 
o  Dew  point  temperature 
o  Wind  speed 
o  Wind  direction 

o  Precipitation  amount  in  one  minute 
o  Precipitation  occurrence 

o  Frozen  precipitation  occurrence  (when  successfully  measured) 
o  Date  of  the  observation 

The  elements  were  transformed  into  categories,  and  dummy  predictors  and 
predictands  were  created.  Table  1  in  columns  3  and  4  shows  the  specific 
categories  defined  for  each  zero-one  dummy  predictor.  Column  1  indicates  the 
dummy  variable  number  while  column  4  gives  the  index  of  that  variable.  One 
dummy  variable  must  be  "left-out"  because  of  mathematical  redundancy. 


2 *3’  Statistical  Analyses 


The  statistical  analyses  which  are  performed  on  these  data  result  from  the 
processing  of  crossproduct  matrices.  The  actual  steps  are  as  follows: 

Step  1.  Compute  the  Z^'Z  and  Z_'Y  crossproduct  matrices  from  the  data 
matrices  Z_  and  Y^. 

Step  2.  Solve  for  A  from  A  =  (Z'Z)  *(Z/Y  )  where  A  is  the 
matrix  of  regress ion^coef fictents  for  making  a  t-minute  forecast. 
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Step.  3.  Solve  for  the  threshold  probabilities  p*  for  making  categorical 
forecasts. 


Derivation  of  the  two  crossproduct  matrices  and  Z^Y  ,  in  step  1  ,  was 
accomplished  by  using  a  pointer  system  which  saved  a  considerable  amount  of 
computer  time.  This  efficiency  is  made  possible  because  of  the  zero-one  nature 
of  the  observations. 


For  the  labeled  predictors  in  Table  2,  Column  4  gives  the  sun  row  of  the 
Z^’Z^  matrix  and  Column  5  the  lowest  cloud  hit  row  of  the  2/  Y  matrix  where 
t  =  10  minutes.  This  column  gives  the  products  between  the  Y  variable  for 
lowest  ceiling  hit  times  each  of  the  88  predictors  over  the  sample  N. 

We  solved  for  the  regression  coefficient  matrix  A  in  step  2  using  the 
Crout  method  (Crout,  1^41).  This  method  does  not  require  solving  for  the 
inverse  matrix,  C^'Z)  ,  but  instead  derives  the  regression  coefficients  by 

first  a  forward  and  then  a  backward  solution.  Avoided  are  many  of  the  computa¬ 
tional  instabilities  encountered  by  inverting  large  matrices.  The  Crout  method 
yields  an  88  x  87  matrix — 88  predictor  coefficients  for  each  of  87  predictands. 


The  lowest  cloud  hit  equation  for  the  A  matrix  at  t  =  If)  minutes  appears 
as  Column  6  in  Table  2. 


2.4.  Nonlinear  Prediction  Approach 

Meteorologists  have  desired  forecast  guidance  that  is  capable  of  predicting 
changes  in  the  weather,  such  as  frontal  passages  and  their  attendant  weather 
variations,  onset  and  discontinuation  of  severe  weather  (types  and  intensities), 
wind  shifts  and  wind  speed  variations,  as  well  as  ceiling  and  visibility  changes 
of  a  critical  nature  for  aviation.  Classical  statistical  approaches  like 
regression  have  not  succeeded  in  completely  satisfying  this  desire,  partly  due 
to  the  additive  nature  of  the  statistical  model  currently  employed.  What  seems 
to  be  needed  is  a  model  which  will  act  in  a  multiplicative  fashion — one  capable 
of  completely  shutting  down  the  prediction  of  an  event  when  the  antecedent 
conditions  warrant.  For  example,  when  it  rains,  it  is  "never"  preceded  1  minute 
before  by  a  clear  sky.  However,  a  statistical-regression  operator  will  fail  to 
turn  off  the  chance  of  rain  fully  if  there  are  other  antecedent  conditions,  say, 
easterly  wind,  high  humidity,  fog,  and  low  visibility-conditions  which  are 
usually  associated  with  future  occurrences  of  rain.  Regression  would  tend  to 
increase  the  probability  of  rain  because  of  each  of  these  elements.  Tn  general 
with  regression,  the  lack  of  any  clouds  would  not  be  enough  to  negate  completely 
the  effect  of  these  other  elements. 


Fortunately,  there  is  a  statistical  model  or  operator  which  possesses  this 
necessary  capability.  The  discrete  likelihood  function  (DLF)  approach  is 
fairly  new  (see  Miller,  1979),  but  the  basis  for  its  existence  is  founded  on 
the  work  of  the  eminent  statistician,  Sir  Ronald  A.  Fisher,  whose  own  work  and 


ideas  on  this  subject  were  derived  from  the  inverse  probability  notions  of 
Bayes  in  the  mid-eighteenth  century.  Basically,  the  concept  is  this:  given 
that  we  observe  a  set  of  current  conditions  of  the  weather,  the  question  to  be 
asked  is  "What  is  the  likelihood  that  these  current  conditions  are  those  that 
would  be  the  conditions  preceding  rain  and,  conversely,  what  is  the  likelihood 
that  these  current  conditions  are  those  that  would  be  the  conditions  preceedlng 
no  rain?”  The  two  likelihoods  are  obtained  by  multiplying  the  conditional 
probabilities  of  each  antecedent  condition  thus  getting  the  joint  probability 
of  the  entire  observation.  It  should  he  emphasized  that  the  presence  of  any 
antecedent  condition  which  is  incongruous  with  an  event  of  interest  (say,  rain) 
wiLl  have  a  dramatic  effect  on  that  likelihood:  it  will  force  the  likelihood 
to  zero.  Such  a  nonlinear  system  would  seem  to  conform  with  meteorologists' 
desires.  Should  the  usual  conditional  probabilities  (posteriors)  be  of 
Interest,  they  can  be  gotten  directly  from  Bayes'  theorem  and  the  climatologi¬ 
cal  frequencies  of  the  possible  events  (priors).  The  linearly  independent 
likelihoods  are  obtained  from  a  set  of  regression  estimated  probabilities 
(REEP)  (see  Miller,  1964).  Empirical  evidence  has  shown  that  rarely  if  ever 
is  a  REEP  probability  of  an  event  <  0  when  the  event  occurs  and  >  1.0  when  it 
does  not  occur.  Certainly  the  situations  arise  when  REEP  forecasts  P  <  0  and 
P  >  1.  However,  truncating  these  REEP  forecasts  to  0  and  1.0,  respectively, 
does  not  seems  to  invalidate  the  reliability  of  the  estimates. 

When  nonlinearity  of  the  type  just  discussed  is  present  in  the  data,  an 
alternative  approach  may  be  more  practical.  It  goes  as  follows: 

o  Test  for  the  equality  of  the  predictand-group  covariance  matrices  £g 
using  Bartlett's  multivariate  test  (see  Morrison,  1967). 

o  If  the  test  suggests  inequality  among  the  £'s,  then  proceed  to  test 
logical  (boolean)  combinations  of  predictor  pairs.  When  certain  pairs 
show  predictive  information  over  and  above  the  basic  set  of  predictors, 
include  them  into  the  GEM  model. 

o  All  of  the  usual  GEM  manipulations  follow  as  before,  such  as  solving  for 
the  coefficient  matrix  A  and  determining  the  threshold  probabilities  for 
making  categorical  forecasts. 


3.  VERIFICATION  RESULTS 


To  demonstrate  the  ability  of  the  GEM  equations  to  predict  at  a  10-minute 
projection,  Brier  scores  (residual  variances)  have  been  computed  on  the 
developmental  sample  for  climatology,  persistence,  and  GEM  for  each  of  the 
predictands  of  interest.  These  are  given  in  Table  3  for  the  specified  dummy 
variables.  The  Brier  score  for  persistence  as  defined  here  uses  only  that 
dummy  element  corresponding  to  the  specific  predictand  dummy.  A  greater 
reduction  (lower  values  are  better)  in  Brier  score  for  persistence  could  have 
been  achieved  if  alt  dummies  of  the  predictand  element  were  used  as 
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predictors.  All  dummies  of  a  predictand  element  were  not  used  as  predictors 
computing  persistence's  Brier  score  for  two  reasons:  a)  the  procedure  is  so 
complex  that  it  would  severely  strain  the  resources  available  to  this  project 
and  h)  more  importantly,  persistence's  function  is  as  a  simple, 
readily-avai lahle  "no  skill”  statistical  control.  The  more  complex  procedure 
is  neither  "readily  available"  nor  simple,  but  a  full-  blown  statistical 
forecasting  procedure  unto  itself.  The  development  of  such  a  procedure  is 
beyond  the  scope  of  this  project. 


3.1.  Comparative  Verification  on  Independent  Data 
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In  May  1983,  independent  data  predictions  were  begun  on  12  weather  elements 
ID,  20,  ...,  60  minutes  in  advance.  A  one  full  year  of  data  (April  1984- 
Apri 1  1985)  represented  the  developmental  sample.  A  complete  display  of  the 
rough lv  100,000  forecasts  made  on  data  from  May  through  August  1985  is  given  in 
the  Appendix  including: 

o  Contingency  tables, 

GEM  versus  actual 
Persistence  versus  actual 
o  Biases, 

o  Percentage  correct, 
o  Heidke  skill  score, 

o  X"  goodness  of  fit  on  the  margins,  and 
o  Threat  score  (critical  success  index) 
for  all  six  projections. 

A  summarization  of  these  scores  is  given  in  Table  4.  For  each  weather 
element  and  each  projection,  the  score  values  are  compared  between  GEM  and 
persistence.  The  better  score  received  a  "G”  if  it  belonged  to  GEM  or  a  "P"  if 
it  belonged  to  persistence.  A  "T”  signifies  a  tie.  The  frequencies  of  these 
designations  are  totaled  to  give  the  overall  comparison. 
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4.  SUMMARY  AND  PLANS 


4.1  Summary 

p 

Overall  GEM  succeeds  in  bettering  persistence  (84  to  42)  and  does  so 
uniformly  over  the  six  projection  periods  of  10,  20,  ...,  60  minutes.  GEM 
manages  to  accomplish  this  through  consistent  superiority  in  the  threat  score — 
an  important  measure  of  the  skill  of  a  procedure  to  predict  rare  events.  GEM,  -V 

however,  seems  to  lack  the  balance  (predicted  frequency  equaling  observed  ^ 

frequency)  for  doing  as  well  in  the  Heidke  skill  score.  Such  a  characteristic  * 

seems  correctable  with  a  more  appropriate  threshold  or  loss  function.  A 
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successful  improvement  here  could  produce  comparisons  that  would  be  even  better 
than  at  present  without  any  new  predictive  information  being  added.  In 
particular,  GEM's  categorical  wind  direction  forecasts  do  not  realize  the 
advantage  they  possess  over  persistence  as  demonstrated  in  the  Brier  score. 


4.2  Plans 


Processing  FAA  data  on  a  microcomputer  has  been  successful.  However,  the 
raw  data  reside  on  400  floppy  disks.  In  the  future  all  of  these  data 
will  be  stored  on  a  single  20  megabyte  hard  disk. 

A  new  software  architecture  is  planned  to  store  the  data  as  bits  (since 
GEM's  variables  are  all  0-1).  Processing  can  be  accomplished  very 
efficiently  using  logical  operations  on  integers. 


Computer  experience  with  the  Kaypro  10  micros  has  been  exceptionally 
good.  They  have  run  24  hours  a  day,  seven  days  a  week  without  a  computer 
failure  at  Sterling,  Virginia.  Similar  reliability  has  been  achieved  with 
the  FAA's  two  research  Kaypro  10's. 

• 

It  is  advisable  to  institute  a  quality  control  of  the  input  data  at  the 
time  it  is  being  collected;  one  that  will  provide  an  alert  for  erroneous 
or  missing  (instrument  malfunction)  information  at  the  source. 


As  things  stand,  the  anticipated  collection  of  Atlantic  City  and 
Indianapolis  data  will  not  be  sensor-variable  compatible  with  Sterling, 
Virginia.  For  example,  cloud  hits  are  taken  4  times  each  minute  at 
Sterling,  Virginia  but  are  taken  only  2  times  each  minute  at  Atlantic 
City,  New  Jersey. 


Further  installations  at  Atlantic  City  and  Indianapolis  will  permit  us  to 
test  whether  the  GEM  equations  can  be  made  general  so  as  to  apply  at  any 
location  without  the  need  to  collect  2  years  of  weather  observations 
first. 


We  will  embark  on  a  full  fledged  AW0S  system  to  predict  10,  20,  ..., 
60  minutes  after  the  observation  as  against  our  previous  efforts  to 
predict  sensor  variables. 


Efforts  will  continue  to  verify  forecasts  based  upon  independent  data 


A  rigorous  investigation  is  planned  to  uncover  and  utilize  nonlinear 
predictive  information.  Such  an  effort  requires  a  computer  upgrade,  the 
new  software  architecture,  and  the  Pascal  language. 


In  conjunction  with  this,  the  National  Weather  Service  plans  to  upgrade 
from  Kaypro  10's  to  Kaypro  286i's. 

A  method  will  be  developed  for  converting  GEM's  probability  forecasts  into 
categorical  forecasts  in  a  more  optimal  fashion  than  what  beta  thresholds 
have  provided. 
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APPENDIX 


This  appendix  has  two  parts.  The  first  part  consists  of  tables  which  are 
referred  to  in  the  text  as  Tables  1-4.  The  second  part  contains  the  actual 
contingency  tables  and  scores  computed  from  the  comparative  verification 
between  GEM  and  persistence.  These  verification  statistics  are  arranged  with 
GEM's  results  for  10,  20,  30,  40,  50,  and  60-minute  projections  followed  by 
persistence  results  for  10,  20,  30,  40,  50,  and  60-minute  projections. 

The  verification  statistics  are  presented  for  the  following  elements: 

Lowest  cloud  hit 
Second  cloud  hit 
Third  cloud  hit 
Fourth  cloud  hit 
Visibility  (miles) 

Station  pressure  (inches  of  Hg) 

Temperature  (°F) 

Dew  point  depression  (°F) 

Wind  speed  (kt) 

Wind  direction  (deg) 

Precipitation  amount  (inches) 

for  the  following  scores: 

Contingency  table 

Biases  (raw  total/column  total) 

Percentage  correct  (hits/total  number  of  forecasts) 

Heidke  skill  score  (hits-chance  hits)/(total-chance  hits) 

Chi-square  goodness  of  fit  on  margins  [(predicted  totals-observed 
totals)  /observed  totals]. 

Threat  score  or  critical  success  Index  [hits/(number  of  forecast 
+  number  of  observed-hits)] 

During  the  period  of  the  present  verification.  May  1985-August  1985,  there 
were  no  yes/no  precipitation  measurements.  The  instrument  was  found  to  have 
been  inoperative.  As  a  result,  all  of  the  yes/no  precipitation  comparative 
verifications  should  be  disregarded.  Apparently,  the  tipping  bucket 
precipitation  amount  measurements  were  not  affected. 


Table  1.  Predictor  and  predictand  categories  which  specify  the  dummy  variables 
used  in  GEM.  Shown  under  the  index  column  are  the  left-out  categories  not 


included 

1  because  of  redundancy. 

Number 

Weather  Element 

Category 

Index 

1 

(Always  unity) 

1 

2 

Lowest  cloud  hit  (00’) 

0  - 

l 

2 

3 

2  - 

4 

3 

4 

5  - 

9 

4 

5 

10  - 

29 

5 

6 

30  - 

60 

6 

7 

61  - 

UNL 

Left  out 

8 

Second  cloud  hit  (00') 

0  - 

1 

7 

9 

2  - 

4 

8 

10 

5  - 

9 

9 

11 

10  - 

29 

10 

12 

30  - 

60 

11 

13 

61  - 

UNL 

Left  out 

14 

Third  cloud  hit  (00') 

0  - 

1 

12 

15 

2  - 

4 

13 

16 

5  - 

9 

14 

17 

10  - 

29 

15 

18 

30  - 

60 

16 

19 

61  - 

UNL 

Left  out 

20 

Fourth  cloud  hit  (00') 

0  - 

1 

17 

21 

2  - 

4 

18 

22 

5  - 

9 

19 

23 

10  - 

29 

20 

24 

30  - 

60 

21 

25 

61  - 

UNL 

Left  out 

26 

Visibility  (miles) 

0  - 

31/64 

22 

27 

1/2  - 

63/64 

23 

28 

1  - 

2  63/64 

24 

29 

3  - 

4  64/64 

25 

30 

5  - 

6  63/64 

26 

31 

7  - 

100 

Left  out 

32 

Station  pressure  (inches  of  Hg) 

0  - 

29.235 

27 

33 

29.236  - 

29.530 

28 

34 

29.531  - 

29.677 

29 

35 

29.678  - 

29.825 

30 

36 

29.826  - 

29.973 

31 

37 

29.974  - 

30.120 

32 

38 

30.121  - 

30.268 

33 

39 

30.269  - 

30.563 

34 

40 

30.564  - 

35.000 

Left  out 

41 

Temperature  (°F) 

—  00  — 

4 

35 

42 

5  - 

14 

36 

43 

15  - 

24 

37 

44 

25  - 

34 

38 

Table  l.  Continued 


Number 


Weather  Element 


Category 


Dew  point  depression  (°F) 


Wind  speed  (kt) 


Wind  direction  (deg) 


Precipitation  amount  (inches) 


Precipitation  occurrence  (Y  or  N) 

Frozen  precipitation  (Y  or  N) 
(when  successfully  measured) 

Month 


3 
4i 

4 
5i 

5 
6i 

6 

7 

8 

95  -  110 
0  -  1 
2-7 
8-15 


Left  out 

48 

49 

50 


16  -  25 

51 

26  -  99 

Left 

0  -  1 

52 

2-9 

53 

10  -  19 

54 

20  -  29 

55 

30  -  99 

Left 

00  -  44 

56 

45  -  89 

57 

90  -  134 

58 

135  -  179 

59 

180  -  224 

60 

225  -  269 

61 

270  -  314 

62 

315  -  359 

Left 

.002  -  .100 

63 

001  -  .0019 

64 

,000  -  .0009 

Left 

Yes 

65 

No 

Left 

Yes 

66 

No 

Lef  t 

January 

67 

February 

68 

March 

69 

April 

70 

May 

71 

June 

72 

July 

73 

August 

74 

September 

75 

Number 


Weather  Element 


Category 


Table  2.  Quantities  derived  for  the  designated  dummy  variables;  the  number  of 
times  each  category  occurred  in  the  sample  O'Z),  the  number  of  times  each 
predictor  occurred  when  it  was  followed  by  the  lowest  cloud  hit  ten  minutes 
later  (ZYZ),  and  the  regression  coefficient  for  each  predictor  when  lowest 
cloud  hit  was  the  predictand  (A). 


Index 


Element 


Table  2.  Continued 


Index 

Element 

Category 

Z  Z 

ZZ'i 

*10 

45 

Temperature  (#F)  cont. 

65  -  74 

72758 

1291 

-.02615 

46 

75  -  84 

41673 

606 

-.02352 

47 

85  -  94 

13241 

195 

-.02291 

48 

Dew  point  depression  (°F) 

0  -  1 

35160 

3078 

.01576 

49 

2  -  7 

158036 

2631 

-.00596 

50 

8-15 

123263 

1319 

-.00516 

51 

16  -  25 

80188 

966 

-.00755 

52 

Wind  speed  (kt) 

0  -  1 

18824 

515 

-.00059 

53 

2-9 

336487 

7027 

.00002 

54 

10  -  19 

61939 

757 

.00786 

55 

20  -  29 

4497 

49 

.00863 

56 

Wind  direction  (deg) 

00  -  44 

33661 

908 

.00732 

57 

45  -  89 

20748 

405 

-.00231 

58 

90  -  134 

30569 

475 

-.00842 

59 

135  -  179 

52449 

1330 

.00084 

60 

180  -  224 

77072 

1639 

-.00207 

61 

225  -  269 

33438 

556 

-.00027 

62 

270  -  314 

95727 

1475 

-.00166 

63 

Precipitation  amount 

.002  -  .100 

60 

1 

-.05164 

64 

(inches) 

.001  -  .0019 

1001 

35 

-.01552 

65 

Precipitation  occurrence 
(Y,N) 

Yes 

17977 

728 

-.00049 

66 

Frozen  precipitation  (Y,N) 

(when  successfully  measured)  Yes 

0 

0 

.00000 

67 

Month 

January 

41688 

827 

-.00622 

68 

February 

37552 

724 

-.00061 

69 

March 

41750 

493 

-.00561 

70 

April 

46108 

646 

-.00416 

71 

May 

25387 

506 

.00000 

72 

June 

30418 

598 

-.00455 

73 

July 

28260 

451 

-.00761 

74 

August 

16613 

268 

-.00082 

75 

September 

35484 

492 

-.00524 

76 

October 

37036 

1023 

-.00028 

77 

November 

40638 

788 

-.00592 

78 

Hour  (LST) 

00  -  00 

34956 

765 

-.00519 

79 

02  -  03 

34553 

860 

-.00930 

80 

04  -  05 

30464 

769 

-.00292 

81 

06  -  07 

34371 

892 

-.00599 

82 

08  -  09 

35620 

896 

.00209 

83 

10  -  11 

35667 

544 

-.00026 

84 

12  -  13 

35739 

431 

-.00680 

85 

14  -  15 

36019 

514 

-.00312 

86 

16  -  17 

36549 

648 

-.00340 

87 

18  -  19 

36459 

673 

.00004 

88 

20  -  21 

36137 

639 

-.00078 

14 
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Table  3.  Brier  scores  for  a  10-minute  forecast  of  each  specified  predictand 
using  the  global  climatological  probability,  the  conditional  persistence 
probability,  and  GEM  based  on  the  developmental  sample  of  468408  cases. 
Dashes  denote  inapplicability. 


Index  Element  Category  Climatology  Persistence  GEM 


1  (Always  unity) 


2 

Lowest  cloud  hit  (00') 

0 

- 

1 

.01940 

.01810 

.01657 

3 

2 

- 

4 

.04848 

.03827 

.03075 

4 

5 

- 

9 

.03551 

.02896 

.02547 

5 

10 

- 

29 

.07399 

.04296 

.03680 

6 

30 

- 

60 

.08152 

.04617 

.04139 

7 

Second  cloud  hit  (00*) 

0 

- 

1 

.05442 

.04379 

.04131 

8 

2 

- 

4 

.02923 

.01349 

.01254 

9 

5 

- 

9 

.02495 

.01571 

.01410 

10 

10 

- 

29 

.06237 

.02857 

.02549 

11 

30 

- 

60 

.07045 

.03622 

.03229 

12 

Third  cloud  hit  (00') 

0 

- 

1 

.02962 

.02614 

.02474 

13 

2 

- 

4 

.02575 

.01134 

.00963 

14 

5 

- 

9 

.02342 

.01435 

.01273 

15 

10 

- 

29 

.05769 

.02610 

.02327 

16 

30 

- 

60 

.06244 

.03164 

.02820 

17 

Fourth  cloud  hit  (00') 

0 

- 

1 

.00143 

.00134 

.00128 

18 

2 

- 

4 

.02208 

.01004 

.00877 

19 

5 

- 

9 

.02225 

.01338 

.01206 

20 

10 

- 

29 

.05226 

.02445 

.02207 

21 

30 

- 

60 

.05344 

.02811 

.02561 

22 

Visibility  (Miles) 

0 

- 

31/64 

.05010 

.03554 

.03419 

23 

1/2 

- 

63/64 

.07108 

.05299 

.05018 

24 

1 

- 

2  63/64 

.03714 

.01902 

.01774 

25 

3 

- 

4  63/64 

.06538 

.03551 

.03262 

26 

5 

- 

6  63/64 

.06322 

.04045 

.03885 

27 

Station  pressure 

(inches  of  Hg) 

0 

- 

29.235 

.00056 

.00013 

.00013 

28 

29.236 

- 

29.530 

.01247 

.00065 

.00064 

29 

29.531 

- 

29.677 

.02501 

.01842 

.01830 

30 

29.678 

- 

29.825 

.09997 

.06048 

.06009 

31 

29.826 

- 

29.973 

.01754 

.01035 

.01029 

32 

29.974 

- 

30.120 

.01977 

.01104 

.01099 

33 

30.121 

- 

30.268 

.01637 

.09078 

.09041 

34 

30.269 

- 

30.563 

.11714 

.00417 

.00415 

35 

Temperature  (°F) 

—  oo 

- 

4 

.00241 

.00008 

.00008 

36 

5 

- 

14 

.00847 

.00131 

.001 30 

37 

15 

- 

24 

.05462 

.00840 

.00822 

38 

25 

- 

34 

.11007 

.01693 

.01641 

39 

35 

- 

39 

.07000 

.01897 

.018  50 

40 

40 

- 

44 

.06987 

.02013 

.01950 

41 

45 

- 

49 

.08046 

.02240 

.02161 

42 

Temperature  (°F)  cont. 

50 

- 

54 

.07818 

.02308 

.02230 

43 

55 

- 

59 

.07711 

.02287 

.02214 

44 

60 

- 

64 

.08436 

.02350 

.02290 

Table  3.  Continued. 


Index 


Element 


45 

46 

47 

48 

49 

50 

51 

52 

53 

54 

55 

56 

57 

58 

59 

60 
61 
62 

63 

64 

65 

66 


67 

68 

69 

70 

71 

72 

73 

74 

75 

76 

77 

78 

79 

80 
81 
82 

83 

84 

85 

86 

87 

88 


Dew  point  depression  (°F) 


Wind  speed  (kt) 


Wind  direction  (deg) 


Precipitation  amount 
(inches) 


(Y,N) 

Frozen  precipitation 
(Y,N)  (when 

successfully  measured) 
Month 


Hour  (LST) 


Yes 

January 
February 
March 
April 
May 
June 
July 
August 
September 
October 
November 
00  -  00 
02  -  03 
04  -  05 
06  -  07 
08  -  09 
10  -  11 
12  -  13 
14  -  15 
16  -  17 
18  -  19 
20  -  21 
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1 

Category 

Climatology 

Persistence 

GEM  £ 

r-/ 

65  -  74 

.14279 

.02181 

.02122  >/ 

75  -  84 

.08891 

.01359 

.01332  -CV 

85  -  94 

.03042 

.00423 

.00412  •/*, 

(  °F) 

0  -  1 

.07640 

.03393 

.03126  & 

2-7 

.23428 

.07114 

.06557  §, 

8-15 

.20686 

.06386 

.06252 

16  -  25 

.15397 

.03901 

.03787 

0  -  1 

.04304 

.03518 

.03385  V 

2-9 

.16165 

.09348 

.08612  .  ;• 

10  -  19 

.12539 

.06327 

.05878 

20  -  29 

.01047 

.00775 

.00744  i 

00  -  44 

.07360 

.05026 

.04757  .vV 

45  -  89 

.04668 

.03327 

.03202 

90  -  134 

.06719 

.04270 

.04116 

135  -  179 

.10876 

.06754 

.06411  ..S 

180  -  224 

.14962 

.07513 

.06896  Jv. 

.05391  (E 

225  -  269 

.07313 

.05558 

270  -  314 

.17534 

.09554 

.08856 

--.V 

.002  -  .100 

.00011 

.00011 

.00011  -\'4 

.001  -  .0019 

.00238 

.00232 

.00226 

rence 

Yes 

.04088 

.01804 

.*  •  1 

.01773  E; 

E 


W 

W'. 

•‘.VJ 


•*.  ■ 


Table  4.  A  summarization  of  the  comparative  skill  between 
GEM,  "G" ,  and  persistence,  ”P“ ,  (ties  are  denoted  by 
"T”)  for  appropriate  scores  at  projections  of  10,  20, 
...,  60  minutes  with  a  tally  for  each.  Percent  correct 
(PC),  Heidke  skill  score  (H),  and  threat  score  (T)  are 
shown  for  the  predictands  listed  in  section  2.2,  except 
precipitation  occurrence.  The  actual  data  from  which 
this  table  was  constructed  is  shown  on  pages  19-91. 
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VERIFYING  OBSERVATION 


HEIDKE  SKILL  SCORE:  .527] 
CHI-SQUARE:  521.32 

THREAT  SCORE:  .2629 
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***GEM  VERSUS  VERIFYING  OBSERVATION  AT  50  MINUTE  INTERVALS*** 


PERCENT  CORRECT:  90.26 
HEIDKF.  SKILL  SCORE:  .5403 

CHI -SQUARE :  130.47 

THREAT  SCORE:  .3042 
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VERIFYING  OBSERVATION 


PERCENT  CORRECT:  47.53% 
IIEIDKE  SKILL.  SCORE:  .3790 

CHI-SQUARE:  121.70 
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PERCENT  CORRECT:  63.75% 
HEIDKF.  SKILL  SCORE:  .5718 

CHI-SQUARE:  2.09 


PRECIPITATION  AMOUNT  (INCHES) 


PERCENT  CORRECT:  100.00% 
HEIDKE  SKILL  SCORE:  1.01 
CHI-SQUARE:  0.00 

THREAT  SCORE:  ****** 


•"■♦PERSISTENCE  VERSUS  VERIFYING  OBSERVATION  AT  20  MINUTE  INTERVALS*** 
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PERCENT  CORRECT:  93.93 
HEIDKE  SKILL  SCORE:  .5916 
CHI-SQUARF,:  2.06 

THREAT  SCORE:  .3859 
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VERIFYING  OBSERVATION 
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♦♦♦PERSISTENCE  VERSUS  VERIFYING  OBSERVATION  AT  30  MINUTE  INTERVALS*** 
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TEMPERATURE  (DEGREES  FAHRENHEIT) 
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PERCENT  CORRECT:  80.95% 
HEIDKE  SKILL  SCORE:  .7416 
CHI-SQUARE:  3.61 


VERIFYING  OBSERVATION 
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PERCENT  CORRECT:  52.49% 
HEIDKF.  SKILL  SCORE:  .4386 
CHI-SQUARE:  10.31 


♦♦♦PERSISTENCE  VERSUS  VERIFYING  OBSERVATION  AT  40  MINUTE  INTERVALS*** 


PERCENT  CORRFCT:  90.42 
HEIDKE  SKILL  SCORE:  .5665 

CUI-SqUARF:  8.66 

THREAT  SCORE:  .3412 
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PERCENT  CORRECT:  76.37% 
HEIDKE  SKILL  SCORE:  .6799 
CHI-SQUARE:  8.60 


VERIFYING  OBSERVATION 


PERCENT  CORRECT:  49.32% 
HEIDKE  SKILL  SCORE:  .4012 
CHI-SQUARE:  16.48 


♦♦♦PERSISTENCE  VERSUS  VERIFYING  OBSERVATION  AT  50  MINUTE  INTERVALS*** 


CHI-SQUARE:  14.02 

THREAT  SCORE:  .2937 
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***fi .RSi STENCE  VERSUS  VERIFYING  OBSERVATION  AT  60  MINUTE  INTERVALS*** 

LOWEST  CLOUD  HIT  (00') 

VERIFYING  OBSERVATION 
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HF.IDKE  SKILL  SCORE:  .5199 
CHI-SQUARE:  22.28 

THREAT  SCORE:  .2472 
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PERCENT  CORRECT:  92.20 
HEIDKE  SKILL  SCORE:  .4777 
CHI-SQUARE:  25.52 

THREAT  SCORE:  .1521 
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PERCENT  CORRECT:  68.83% 
HEIDKE  SKILL  SCORE:  .5787 
CHI-SQUARE:  27.26 
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HEIDK.:  SKILL  SCORE:  .3603 
CHI-SQUARE:  t-U. 31 


